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Finite-state complexity is a variant of algorithmic information theory obtained by replacing Turing 
machines with finite transducers. We consider the state-size of transducers needed for minimal de- 
scriptions of arbitrary strings and, as our main result, show that the state-size hierarchy with respect 
to a standard encoding is infinite. We consider also hierarchies yielded by more general computable 
encodings. 
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1 Introduction 

Algorithmic information theory (7J uses the minimal size of a Turing machine that outputs a string 
x as a descriptional complexity measure. The theory has produced many elegant and important results; 
however, a drawback is that all variants of descriptional complexity based on various types of universal 
Turing machines are incomputable. Descriptional complexity defined by resource bounded Turing ma- 
chines has been considered in 0]], and, at the other end of the spectrum, lie models based on context-free 
grammars or finite automata. 

Grammar-based complexity measures the size of the smallest context-free grammar generating a 
single string. This model has been investigated since the 70's, and recently there has been renewed 
interest due to applications in text compression and connections with Lempel-Ziv codings, see e.g. lfl2l 
[T3l ; a general overview of this area can be found in ifTTl . The automatic complexity of a string ifTTl is 
defined as the smallest number of states of a DFA (deterministic finite automaton) that accepts x and 
does not accept any other string of length |x|. Note that a DFA recognizing the singleton language 
{x} always needs |jc| + 1 states, which is the reason the definition considers only strings of length \x\. 
Automaticity HI [HI is an analogous descriptional complexity measure for languages. The finite-state 
dimension is defined in terms of computations of finite transducers on infinite sequences, see e.g. |2l[9]]. 

The NFA (nondeterministic finite automaton) based complexity of a string (H can also be viewed 
as being defined in terms of finite transducers that are called "NFAs with advice" in (H. However, the 
model allows the advice strings to be over an arbitrary alphabet with no penalty in terms of complexity 
and, as observed in O, consequently the NFAs used for compression can always be assumed to consist 
of only one state. 

The finite-state complexity of a finite string x was introduced recently O in terms of a finite trans- 
ducer and a string p such that the transducer on input p outputs x. Due to the non-existence of universal 
transducers, the size of the transducer is included as part of the descriptional complexity measure. We 
get different variants of the measure by using different encodings of the transducers. 
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In our main result we establish that the measure results in a rich hierarchy in the sense that there is 
no a priori upper bound for the number of states used by transducers in minimal descriptions of given 
strings. The result applies to our standard encoding, as well as to any other "reasonable" encoding where 
a transducer is encoded by listing the productions in some uniform way. 

By the state-size hierarchy we refer to the hierarchy of languages L<„„ m > 1, consisting of strings 
where a minimal description uses a transducer with at most m states. We show that the state-size hierarchy 
with respect to the standard encoding is infinite; however, it remains an open question whether the 
hierarchy is strict at every level. 

In a more general setting, the definition of finite-state complexity (6l allows an arbitrary computable 
encoding of the transducers, and properties of the state-size hierarchy depend significantly on the partic- 
ular encoding. We establish that, for suitably chosen computable encodings, every level of the state-size 
hierarchy can be strict. 

2 Preliminaries 

If X is a finite set then X* is the set of all strings (words) over X, with e denoting the empty string. The 
length of x G X* is denoted by \x\. We use C to denote strict set inclusion. 

For all unexplained notions concerning transducers we refer the reader to SHU- In the following, by 
a transducer we mean a (left) sequential transducer Q, also called a deterministic generalised sequential 
machine lfT8l . where both the input and output alphabets are {0, 1}. The set of all transducers is J^gsm- 

A transducer T G ^gsm is denoted as a triple T = (Q,qo,A) where Q is the finite set of states, 
<7o £ Q is the start state, (all states of Q are considered to be final), and 

A: 0x {0,1} {0,1}* (1) 

is the transition function. When a transducer is represented as a figure, each transition A(q,i) = (p,w), 
liP € <2> i € {0, 1}, w G {0, 1}*, is represented by an arrow with label i/w from state q to state p, and i 
(respectively, w) is called the input (respectively, output) label of the transition. By the (state) size of T, 
size(r), we mean number of states in the set Q. 

The function {0, 1}* — > {0, 1}* computed by the transducer T is, by slight abuse of notation, also 
denoted by T and defined by T(e) = £, T(xa) = T(x) ■ 7t 2 (A(S(q ,x),a)), for x G {0, 1}*, a G {0, 1}. 
Here 7ij, i = 1,2, are the two projections on Q x {0, 1}*, and 8 : Q x {0, 1}* — > Q is defined by 8(q,e) = 
q,8(q,xa) = K\(A(8(q,x),a)), q G Q, x G {0, 1}* a G {0, 1}. 

By a computable encoding of all transducers we mean a pair S = (Ds,fs) where D$ C {0, 1}* is a 
decidable set and f$ : D$ — > ^dgsm is a computable bijective mapping that associates a transducer to 
each a GD 5 Q 

We say that S is a polynomial-time (computable) encoding if D$ G P and for a given a G D$ we 
can compute the transducer T$ G ^dgsm in polynomial time. We identify a transducer T G ^dgsm with 
its transition function £[]), and the set of state names is always {1,...,|<2|} where 1 is the start state. 
By computing the transducer T% we mean an algorithm that (in polynomial time) outputs the list of 
transitions (corresponding to (Q]), with state names written in binary) of T^. 

Next we define a fixed natural encoding So of transducers that we call the standard encoding. For 
our main result we need some fixed encoding of the transducers where the length of the encoding relates 

'in a more general setting the mapping fg may not be injective (for example, if we want to define D$ as a regular set ||6l). 
however, in the following we restrict consideration to bijective encodings in order to avoid unnecessary complications with the 
notation associated with our state-size hierarchy. 
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in a "reasonable way" to the lengths of the transition outputs. We encode a transducer as a binary string 
by listing for each state q and input symbol i G {0, 1} the output and target state corresponding to the 
pair (q,i), that is, A(q,i). Thus, the encoding of a transducer is a list of (encodings of) states and output 
strings. For succinctness, in the list we omit (that is, replace by e) the states that correspond to self-loops. 

By bin(/) we denote the binary representation of i > 1. Note that for all i > 1, bin(z') always begins 
with a 1. For v = Vi • • • v m , v,- G {0, 1}, i = 1, . . . ,m, we use the following functions producing self- 
delimiting versions of their inputs (see HI): = viCH^O- • • v m _i0v„,l and v° = (lv) 1 ", where ~~ is the 
negation moiphism given by = 1, 1 = 0. It is seen that (v^ = 2\v\, and |v°| = 2\v\ +2. 

We define the set D$ to consist of all strings of the form 

<T = bin(ii)*-vt -binfe^-vl-.-bin^)*- v|„, (2) 
where 1 < i t <n,v t G {0,1}*,? = 1 , . . . , 2n, and 

binW^I^^^^' ,1<K2, 
\ e if h = f §1 . 

A string a as in © encodes the transducer = ({1, . ..,«}, 1, A), where A(j,k) = 
(hj-i+k,V2j-i+k), j ; = 1, k G {0,1}. Note that in ©, bin(j f )* = e if the corresponding transi- 
tion of A is a self-loop. 

Now we define the standard encoding So as the pair (Ds , fs ) where fs associates to each a € So 
the transducer T^ as described above. It can be verified that for each T G 2?dgsm there exists a unique 
a G Ds such that T = , that is, T and have the same transition function. The details of verifying 
that ^ rjf when 0\ / 02 can be found in Q. For T G 3?dgsm> the standard encoding of T is the 
unique a G Ds such that T = . The standard encoding So is a polynomial-time encoding. 

Note that using a modification of the above definitions it is possible to guarantee that the set of all 
legal encodings of transducers is regular J6j - this is useful e.g., for showing that the non-existence 
of a universal transducer is not caused simply by the fact that a finite transducer cannot recognize legal 
encodings of transducers. More details about computable encodings can be found in O, including binary 
encodings that are more efficient than the standard encoding. 



3 Finite-state complexity 

In the general form, the transducer based finite-state complexity with respect to a computable encoding 
S of transducers in $dgsm is defined as follows O. 

We say that apair (T$,p), o £D s ,p£ {0, 1}*, defines the string* G {0, 1}* provided that T§(p) =x; 
the pair (T$,p) is called a description of x. As the pair (T$,p) is uniquely represented by the pair (o,p) 
we define the size of the description (T§,p) by 

||(r ff 5 ,p)|| s =|a| + |^|. 

We define the finite-state complexity of a string x G {0, 1}* with respect to encoding S by the formula: 

C s {x)= inf {\o\ + \p\;Tl(p)=x\. 

We will be interested in the state-size, that is, the number of states of transducers used for minimal 
encodings of arbitrary strings. For m > 1 we define the language L s <m to consist of strings x that have a 
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minimal description using a transducer with at most m states. Formally, we write 

L| m ={ *€{0,1}* | (3o£D s ,p€{0,iy)TS(p)=x, 
\g\ + \p\ =C s (x), size(rj) <m}. 

By setting L< = 0, the set of strings x for which the smallest number of states of a transducer in a 
minimal description of x is m can then be denoted as 

L 5 =m = L< m — L| m _ ! , m > 1 . 

Also, we let Ln ,„ denote the set of strings x that have a minimal description in terms of a transducer 
with exactly m states. Note that L s =m C Ln but the converse inclusion need not hold, in general, 
because strings in Ln ,„ may have other minimal descriptions with fewer than m states. 

In the following, when dealing with the standard encoding So (introduced in Section |2j) we write, for 
short, T a , 1 1 (T,p) 1 1, C and L< m , L =m , L 3imn , n , m > 1, instead of T*°, \ \ (T,p) \ \ So , C So and L% n , L s J m , L S ^ m , 
respectively. The main result in section|4]is proved using the standard encoding; however, it could easily 
be modified for any "naturally defined" encoding of transducers, where each transducer is described by 
listing the states and transitions in a uniform way. For example, the more efficient encoding considered 
in (6l clearly satisfies this property. On the other hand, when dealing with arbitrarily defined computable 
encodings S, the languages L s <m , m > 1, obviously can have very different properties. In section [5] we 
will consider properties of more general computable encodings. 

The fmite-state complexity with respect to an arbitrary computable encoding S is computable O 
because for given x, \oi\ + \x\ gives an upper bound for Cs(x) where Oi € 5 is an encoding of the one- 
state identity transducer. An encoding of the identity transducer can be found from an enumeration of 
strings in S, and after this we can simply try all transducer encodings and input strings up to length 
|<7i | + |je|. Hence "inf" can be replaced by "min" in the definition of C5. 

Proposition 3.1 For any computable encoding S, the languages L s <m , m > 1, are decidable. 

We conclude this section with an example concerning the finite-state complexity with respect to the 
standard encoding. 

Example 3.1 Define the sequence of strings 

w m = 1010 2 10 3 l-...-0 m ^ 1 10 m l, m>\. 

Using the transducer T\ of Figure Q] we produce an encoding of W99. Note that \w^ \ = 5050. 

With the encodings of the states indicated in Figure 1, T\ is encoded by a string Oy € Sq of length 352. 
Each number < i < 99 can be represented as a sum of, on average, 3.18 numbers from the multi-set 
{1,5, 10, 18,25,50} ifTSl . Thus, when we represent W99 in the form ^(pgg), we need on average at most 
6-3.18 symbols in 7799 to output each substring ! , < i < 99. (This is only a very rough estimate since 
it assumes that for each element in the sum representing i we need to make a cycle of length six through 
the start state, and this is of course not true when the sum representing i has some element occurring 
more than once.) Additionally we need to produce the 100 symbols "1", which means that the length of 
/?99 can be chosen to be at most 2008. Our estimate gives that 

||(r ffl ,/?99)|| = |ai| + M=2360, 
which is a very rough upper bound for C(w99). 
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Figure 1: The transducer T\ for Example |3. II 

The above estimation could be improved using more detailed information from the computation of 
the average from lfl31 . Furthermore, lTT5l gives other systems of six numbers that, on average, would give 
a more efficient way to represent numbers from to 99 as the sum of the least number of summandsH 
These types of constructions can be seen to hint that computing the value of finite-state complexity 
may have connections to the so-called postage stamp problems considered in number theory, with some 
variants known to be computationally hard lfT0l[T4"1 . It remains open whether computing the function C 
(corresponding to the standard encoding) is NP-hard, or more generally, whether for some polynomial- 
time encoding S, computing Cs is NP-hard 0. 

4 State-size hierarchy 

We establish that finite-state complexity is a rich complexity measure with respect to the number of states 
of the transducers, in the sense that there is no a priori upper bound for the number of states used for 
minimal descriptions of arbitrary strings. This is in contrast to algorithmic information theory, where the 
number of states of a universal Turing machine can be fixed. 

For the hierarchy result we use the standard encoding So- The particular choice of the encoding is not 
important and the proof could be easily modified for any encoding that is based on listing the transitions 
of a transducer in a uniform way. However, as we will see later, arbitrary computable encodings can 
yield hierarchies with very different properties. 

Theorem 4.1 For any n £ IN there exists a string x n such that x n ^ L< n . 

Proof. Consider an arbitrary but fixed n € IN. We define 2n + 1 strings of length 2n + 3, 

Ui = 10'l 2 " +2 ~', i=l,...,2n + l. 
2 In 1151 it is established that 18 is the optimal value to add to an existing system of {1,5, 10,25,50}. 
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For m > 1 , we define 

x n{ m ) = u \ u 2 "' u 2n+\- 

Let (T a ,p) be an arbitrary encoding of x n (m) where size(r CT ) < n. We show that by choosing m to be 
sufficiently large as a function of n, we have 

II(W)II>y- < 3 > 

The set of transitions of T a can be written as a disjoint union Q\ U 62 U 63, where 

• d\ consists of the transitions where the output contains a unique m, 1 < i < 2n + 1, as a substring H 
that is, for any 7 ^ i, uj is not a substring of the output; 

• 62 consists of the transitions where for distinct 1 < i < j < 2n + 1, the output contains both m, and 
uj as a substring; 

• 63 consists of transitions where the output does not contain any of the w;'s as a substring, i = 
1,..., 2/1+1. 

Note that if a transition a £ 63 is used in the computation T a {p), the output produced by a cannot 
completely overlap any of the occurrences of m/'s, / = 1, . . . ,2n + 1. Hence 

a transition of 63 used by r CT on p has output length at most An + 4. (4) 

Since T a has at most n states, and consequently at most 2n transitions, it follows by the pigeonhole 
principle that there exists 1 < k < 2n + 1 such that is not a substring of any transition of 6\ . We 
consider how the computation of T a on p outputs the substring u™ of x n {m). Let zi, ■ ■ ■ , Z r be the 

minimal sequence of outputs that "covers" u™ . That is, z\ (respectively, z r ) is the output of a transition 

2 2 

that overlaps with a prefix (respectively, a suffix) of uf and uf is a substring of z\ ■ ■ -z r - 
Define 

2/ = {1 < 7 < r I Zj is output by a transition of 0,}, / = 1,2,3. 

By the choice of k we know that Si = 0. For 7 £ S2, we know that the transition outputting z ; - can be 
applied only once in the computation of T a on p because for i < j all occurrences of u\ as substrings of 
x n (m) occur before all occurrences of Uj. Thus, for j G S2, the use of this transition contributes at least 
2 • |zy I to the length of the encoding 1 1 (T a , p) \ \ . 

Finally, by ((U, for any j € S3 we have \zj\ < An + 4 < 2\uk\. Such transitions may naturally be 
applied multiple times, however, the use of each transition outputting Zj, j € S3, contributes at least one 
symbol to p. 

Thus, we get the following estimate: 

ll(W)||> L2.|z 7 -| + |2 3 |>^ = ^- 

2 

To complete the proof it is sufficient to show that, with a suitable choice of m, C(x n (m)) < The 
string x n (m) can be represented by the pair {T\,p\) where 7\ is the 2«-state transducer from Figure|2]and 
Pl = ( m l) 2 "- 1 m l m . 



3 By a substring we mean a "continuous substring". 
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Figure 2: The transducer Ti from the proof of Theorem 14. II 



Each state of T\ can be encoded by a string of length at most [~log 2 (2n)~|, so (recalling that in the 
standard encoding each transition output v contributes |v°| = 2|v| + 2 to the length of the encoding and 
each binary encoding u of a state name that is the target of a transition that is not a self-loop contributes 
2\u\ to the length of the encoding) we get the following upper bound for the length of a string 0\ G Sq 
encoding T\ : 

|oi| < (8n 2 + 16n + 8)m+(4n-2)([log 2 (2n)] + 1). 



Noting that \p\ \ = (2n + \)m +2n — 1 we observe that 



C(x„{m)) < \\(T ai ,pi] 



\ a i\ + \Pi\ < y 



(5) 



for example, if we choose m = I6n 2 + 36n + 19. This completes the proof. 



As a corollary we obtain that the sets of strings with minimal descriptions using a transducer with at 
most m states, m > 1, form an infinite hierarchy. 







Corollary 4.1 For any n ^ 1, there exists effectively k n > 1 such that L< n C .L< n _|_£„ 

We do not know whether all levels of the state-size hierarchy with respect to the standard encoding 
are strict. Note that the proof of Theorem 14.11 constructs strings x n (m) that have a smaller description 
using a transducer with 2n states than any description using a transducer with n states. We believe that 
(with m chosen as in the proof of Theorem 14. II ) the minimal description of x n (m), in fact, has 2n states, 
but do not have a complete proof for this claim. The claim would imply that L<„ is strictly included 
in L<2„, n > 1. In any case, the construction used in the proof of Theorem 14. 1 1 gives an effective upper 
bound for the size of k n such that L<„ C £< n +jfc„ . because the estimation (f5]) (with the particular choice 
for m) implies also an upper bound for the number of states of a transducer used in a minimal description 
of x n {m). 

The standard encoding is monotonic in the sense that adding states to a transducer or increasing the 
lengths of the outputs, always increases the length of an encoding. This leads us to believe that for any n 
there exist strings where the minimal transducer has exactly n states, that is, for any n > 1, L =n ^ 0. 

Conjecture 4.2 L<„ C L<„+i, for all n> I. 

By Proposition 13 . 1 1 we know that the languages L<„ are decidable. Thus, for n > 1 such that L =n ^ 0, 
in principle, it would be possible to compute the length l n of shortest words in L =n . However, we do 
not know how £„ behaves as a function of n. Using a brute-force search we have established @ that all 
strings of length at most 23 have a minimal description using a single state transducer. 



4 Note that here "C" stands for strict inclusion. 
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Open problem 1 What is the asymptotic behavior of the length of the shortest words in L =n as a function 
ofn? 

Also, we do not know whether there exists x 6 {0,1}* that has two minimal descriptions (in the 
standard encoding) where the respective transducers have different numbers of states. This amounts to 
the following: 

Open problem 2 Does there exist n> I such that L=„ ^ L^ mmn ? 

5 General computable encodings 

While the proof of Theorem K.ll can be easily modified for any encoding that, roughly speaking, is based 
on listing the transitions of a transducer, the proof breaks down if we consider arbitrary computable 
encodings 5. Note that the number of transducers with n states is infinite and, for arbitrary computable 
S, it does not seem easy, analogously as in the proof of Theorem 14. 1[ to get upper and lower bounds for 
Cs(x n (m)) for suitably chosen strings x„(m). We do not know whether there exist computable encodings 
for which the state-size hierarchy collapses to a finite level. 

Open problem 3 Does there exist n>\ and a computable encoding S n such that that, for all k>l, 
jS„ tS„ <? 

On the other hand, it is possible to construct particular encodings for which every level of the state- 
size hierarchy is strict. 

Theorem 5.1 There exists a computable encoding Si such that 

C L< n , for each n>\. 

Proof. Let pu i = 1,2,..., be the ith prime. We define an rc-state (n > 1) transducer T n = 
({l,...,n}, 1,A„) by setting by A„(1,0) = (1,0*), A n (j,0) = (i,e), 2 < i < n, A n (j,l) = (j + l,e), 
1 < j < n — 1, and A„(«, 1) = (n,e). 

In the encoding Si we use the string o n = bin(«) to encode the transducer T n , n > 1. Any transducer 
T that is not one of the above transducers T n , n > 1, is encoded in Si by a string 0-e,e£ {0, 1}*, where 
\e\ is at least the sum of the lengths of outputs of all transitions in T . This condition is satisfied, for 
example by choosing the encoding of T in Si to be simply concatenated with the standard encoding of 
T. 

Let m > 1 be arbitrary but fixed. The string Pm has a description (7^,0) of size [logm] + 1, where 
a m <E Si encodes T m and the transducer T^ n has m states. We show that (0 Pm ) = [logm] + 1. 

By the definition of the transducers T n , for any w £ {0, 1}*, T n (w) is of the form k ' Pn , k>0. Thus, 
Pm cannot be the output of any transducer T n ,n ^ m. 

On the other hand, consider an arbitrary description (T^ 1 , w) of the string Pl " where T^' is not any 
of the transducers T n , n > 1. Let x be the length of the longest output of a transition of T a . Thus, 
x ■ \w\ > p m . By the definition of Si we know that \a\ > x+ 1, and we conclude that 

||(r*,w)|| 5l = |<T| + H> [togml+1. 

We have shown that, in the encoding Si , the unique minimal description of Pm uses a transducer with m 
states, which implies Pm € ZA' m . ■ 
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The encoding S\ constructed in the proof of Theorem [5J] is not a polynomial-time encoding because 
T n has an encoding of length 0(logn), whereas the description of the transition function of T n (in the for- 
mat specified in Section |2]) has length £l(n • log 72). Besides the above problem Si is otherwise efficiently 
computable and using standard "padding techniques" we can simply increase the length of all encodings 
of transducers in S\ . 

Corollary 5.1 There exists a polynomial time encoding S\ such that 



Proof. The encoding S[ is obtained by modifying the encoding Si of the proof of Theorem 15.11 as 
follows. For n > 1, T„ is encoded by the string o n = bin(n) 1 " • 1". Any transducer T that is not one of the 
transducers T„,n> 1, is encoded by a string • w where \w\ > 2 X and x is the sum of the lengths of outputs 
of all transitions of T. If a is the standard encoding of T, for example, we can choose w = ■ l 2 " . 

Now \a„\ is polynomially related to the length of the description of the transition function of T n , 
n>\, and given a„ the transition function of T n can be output in quadratic time. For transducers not of 
the form T„, n > 1, the same holds trivially. 

Essentially in the same way as in the proof of Theorem 15. II we verify that for any m > 1, the string 

S' 

Pm has a unique minimal description (TJ,0), where a' € S\ is the description of the m-state transducer 
T m . The same argument works because, the encoding of any transducer T in S[ is, roughly speaking, 
obtained from the encoding a of T in Si by appending 2^ symbols 1. ■ 

There exist computable encodings that allow minimal descriptions of strings based on transducers 
with different numbers of states. Furthermore, the gap between the numbers of states of the transducers 
used for different minimal descriptions of the same string can be made arbitrarily large, that is, for any 
n < m we can construct an encoding where some string has minimal descriptions both using transducers 
with either norm states. The proof uses an idea similar to the proof of Theorem l5.ll 

s s 

Theorem 5.2 For any 1 < n < m, there exists a computable encoding S„ m such that L q " '" n L=„ 7^ 0. 

Note that the statement of Theorem [5T2] implies that L=^ 7^ L^" ,m Again, by padding the encodings 
as in Corollary 15. 11 the result of Theorem l5.2l can be established using a polynomial-time encoding. 

6 Conclusion 

As perhaps expected, the properties of the state-size hierarchy with respect to the specific computable 
encodings considered in section [5] could be established using constructions where we added to transduc- 
ers additional states without changing the size of the encoding. In a similar way various other properties 
can be established for the state-size hierarchy corresponding to specific (artificially defined) computable 
encodings. The main open problem concerning general computable encodings is whether it is possible 
to construct an encoding for which the state-size hierarchy collapses to some finite level, see Problem [3] 
As our main result we have established that the state-size hierarchy with respect to the standard 
encoding is infinite. Many interesting open problems dealing with the hierarchy with respect to the 
standard encoding remain. In addition to the problems discussed in section 0J we can consider various 
types of questions related to combinatorics on words. For example, assuming that a minimal description 
of a string w needs a transducer with at least m states, is it possible that w 2 has a minimal description 
based on a transducer with less than m states? 

Conjecture 6.1 Ifw G L- m (m> I), then for any k> 1, w k L< m -\. 




<n-\ 



cl: 




for each n > 1 . 
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